# Parameter-shared architecture

ALBERT XXLarge v2 is a large language model pre-trained with masked language modeling objectives, featuring a parameter-shared Transformer architecture with 12 repeated layers and 223 million parameters.

ALBERT XLarge v2 is an English pretrained model based on the Transformer architecture, employing parameter-sharing mechanisms to reduce memory usage, trained with masked language modeling and sentence order prediction objectives.